Dialectal Arabic Orthography-based Transcription

The present paper describes the experience gained at LDC in the collection and transcription of conversational dialectal Arabic. The paper will cover the following: (a) Arabic language background; (b) objectives. principles, and methodological choices of dialectal Arabic transcription, (c) design features of LDC‟s „Arabic MultiDialectal Transcription Tool‟ (AMADAT) and metalanguage transcription issues, and finally (d) a summary description of the technical specifications, process, current results and issues of the EARS Levantine Arabic Conversational Telephone Speech Collection. 1.0 Introduction: Arabic Language Background The Arabic language is a „linguistic continuum‟ (Hymes, 1973) with two major poles representing an Arabic Standard, the language of most written and formal spoken discourse, and a collection of related Arabic dialects, which are mainly spoken and which present significant phonological, morphological, syntactic, and lexical differences among themselves and when compared to the standard written forms. This situation, usually referred to as „diglossia‟ (Ferguson, 1959), presents some challenging problems for Arabic spoken language technologies, including corpus creation to support Speech-to-Text (STT) systems, since the spoken Arabic dialects are not officially written and have no standardized writing in spite of growing but still relatively small and not wholly conventionalized web activities. A significant amount of linguistic variation occurs and produces many variant forms which are difficult to identify and regroup. 1.1 Arabic Dialectal Variation The diglossic situation described above represents a significant linguistic distance between all Arabic dialects and the „fusha,‟ commonly identified as „Modern Standard Arabic‟ (MSA), though the latter term does not cover all features of the former. This linguistic distance is characterized by substantial phonological, morphological, and lexical variation. Arabic dialectal variation is significant not only between major dialects, (e.g. Egyptian, Levantine, Gulf, or Maghrebi) but also between the regional variants of any major dialect (e.g. Northern and Southern Levantine) and even between the subdialects of any regional variant. Since important sound change has occurred in all Arabic dialects, the complexity of the above situation resides in the existence of significant differences between the phonologies of the various Arabic dialects. In Levantine Arabic (LA), for instance, the sound /q/ is pronounced /q/ but also /‟/, /g/ and /k/. In Egyptian Arabic, /?/ replaces /q/ with few lexical exceptions and not in all subdialects. In Sudanese Arabic, MSA /q/ is replaced by /g/ and sometimes the uvular [  ]. All of the above creates confusion which needs to be addressed and taken into account in any dialectal transcription task. 1.2 Nature of the Dialectal Arabic Transcription